So welcome back to deep learning and today I want to talk about more visualization techniques

but actually I want to start motivating why we need visualization techniques in the next

couple of minutes.

Okay, here we go.

Visualization and Attention Mechanisms Part 2.

In this video, we will be looking at the

implementation of parameters and the first thing is the motivation.

So the networks learn a representation of the training data, but the question is, of course,

what happens with the data in our network?

And this is really an important thing, and you should really care about this,

because it's very useful to investigate unintentional and unexpected behavior.

And one thing that I really want to highlight here are adversarial examples.

I want to show you why your network performs really well in the lab, but it fails in the wild.

So this is also an important thing that you want to be solving.

Then you can figure out potential causes for this, because if you look into these visualization techniques,

they will help you to identify, focus on wrong types of features, noise properties, and things like that.

So we'll have a couple of examples in the next videos.

I want to show you some anecdotal examples, for example, the identification of tanks in photos.

So this is actually an example from Neil Fraser's website, a Google developer.

And I'm not entirely sure if this really happened or whether this is just an urban legend.

So the legend goes like this.

People in the Pentagon wanted to train a neural network to identify tanks on images.

What could you do in order to construct such a set?

Well, you go out there and then you take images of tanks and then you take images of non-tank situations.

Well, typically you would expect them to be in some scenery.

So you go out into the forest and take some pictures.

And then, of course, you have to get some pictures of tanks and tanks you typically find on a battlefield.

And there's, you know, smoke around and mud and dirty and gritty.

So you collect your images of tanks and then you have maybe 200 of the forest images and 200 of the tank images.

You go back to your lab and then you train your deep neural network, or maybe not so deep if you only have this very small data set.

And you go ahead and you get an almost perfect classification rate.

So everybody's very happy.

It seems that your system is working really well.

So I have two examples here on this slide.

And you will say, yeah, I've solved the problem.

So let's build a real system from this.

And this will warn us of tanks.

And they built the system and they realized it didn't work at all in practice.

They actually had a recognition rate of approximately 50 percent in this two class problem.

This means this is approximately random guessing.

So what could have possibly gone wrong?

Well, if you look at those images, you can see that all of the forest images, they have essentially all been taken on sunny, nice weather days.

And then, of course, you see that the tanks, they have been taken on days that are more cloudy.

And, you know, they are not so good weather conditions, of course, when you see the tanks, because they do all kinds of things.

They fire, there's grenades around.

And of course, this means that there will be smoke and other things happening.

So what the system essentially learned is not to identify tanks, but it took the shortcut.

And here the shortcut is that you try to detect the weather.

So if you have a blue sky, good weather conditions, very few noise in the image, then it's potentially a non-tank image.

And if you have noise and bad lighting conditions, then it's potentially a tank image.